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Abstract 

Recently Miller and Sofer have presented a new nonparametric method for estimating the failure 
rate of a software program. The method is based on the complete monotonicity property of the failure 
rate function, and uses a regression approach to obtain estimates of the current software failure rate. 

This paper extends this completely monotone software model and demonstrates how it can also provide 
long-range predictions of future reliability growth. Preliminary testing indicates that the method is 
competitive with parametric approaches, while being more robust. 

1 Introduction 

Suppose a program is executed for a length of time T. During this time, n bugs are detected and removed 
when they manifest themselves as failures. The successive failures occur at times 

0 < fi < < 2 < .. < t n < T. (1) 

When bugs are corrected without introducing new faults, the program evolves into a more reliable program, 
hence the term “reliability growth”. Given the past software data (1) we would like to make various statistical 
inferences concerning the current and future reliability of the software. In particular we are interested in the 
number of failures expected over some future horizon, the present failure rate, and the future failure rate 
after an additional specified time of debugging. 

Over the years, many competing models for software reliability growth have been developed. These models 
include those by Duane(1964), Jelinski and Moranda(1972), Goel and Okumoto(1979), Littlewood(1981) 
and Musa and Okumoto(1984). These are all parametric models. It is interesting to note that all the 
above models have a common property: complete monotonicity of the failure rate function. Let be 

the (random) number of failures observed in [0,t], and let M(t) = E(N(t)) be the expected number of 
failures. M(t) is known as the mean function. The intensity function of the point process {iV(t), 0 < £} is 
r(t) = dM{t)fdt, 0 < t. The function r(t) is also referred to as the failure rate of the process. A function r(-) 
is completely monotone if and only if it has derivatives of all orders, and they alternate in sign as follows: 

>°. 4 >°> ? = ( 2 ) 

v dti 

Miller(1986) has shown that virtually all completely monotone functions can occur as intensity functions of 
reliability growth point processes. Thus, a general approach to software reliability growth modelling should 
include the entire class of completely monotone intenisities. 

Miller and Sofer(1985) have previously introduced a nonparametric model for software reliability growth 
which is based on complete monotonicity of the failure rate function. The method uses a regression approach 
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to obtain estimates of the current software failure rate. Miller and Sofer( 1986a) show that this method often 
Kives estimates which have a lower bias than those of certain (widely-used) nonparametric methods; using 
Monte Carlo simulated failure data, these “completely monotone regression” estimates of current failure rate 
are also shown to be more robust than the estimates based on parametric models. Chan( 1986) has estimated 
the distribution of time until next failure for real data using completely monotone regression estimates of 
current reliability: he starts with a raw estimate which is an exponential distribution with the estimated 
current failure rate and then “adapts” it to a more general distribution using the procedure of Litt ewood 
and Keiller(1984). Chan then evaluates these estimates using criteria of Abdel-Ghaly, Chan and Littlewood 
(1986). This study shows that the completely monotone regression approach gives good estimates which are 

more robust than estimates from parametric models. ... . 

This paper extends the completely monotone software model by developing a method for providing long- 
range predictions of future reliability growth, based on the model. The paper derives upper and lower bounds 
on extrapolations of the failure rate function and the mean function into the future. These are then utilised 
to obtain estimates for the future software failure rate and the expected future number of failures. 


2 Problem Formulation 


Consider the failure data as in (1). Our goal is to find a completely monotone rate function and/or the 
associated mean function which best fits the data. (We note here that the mean function does not strictly 
satisfy the complete monotonicity property; rather, M(t) is a nonegative function whose derivative dM[t)Jdt 
is a completely monotone function). Our approach will be to obtain an initial raw estimate for the required 
function from the data, and then to smooth it by fitting a completely monotonic function which is closest 

to it in the least squares sense. t , , . 

A natural raw estimate M(t) for the mean function is a piecewise linear function with breakpoints at 

t u i = 1, . . . , n, such that M(t,) = i. Analytically, this function is given by 



i + {t - ti)/(t i+ 1 -k) U<t< tj+i; * = 0, . . 

n + 6(t-t n )/{T-t n ) t n <t <T. 


n — 1 


( 3 ) 


The second term in the final interval reflects the absence of a failure in the period (t n) T]. The choice of 
is somewhat arbitrary, with higher values tending tending to give more conservative estimates. In this work 
we consider values of 0.0, 0.5 and 1.0 for 6, however one can argue for and against any particular value. 

In practice, it is necessary to discretize the problem of finding a completely monotone function to the 
mathematically more tractable problem of finding a finite set of points along that function. The most 
plausible and straightforward approach is to consider discrete time points which are equally spaced. We 
thus divide the time interval [0, T] to k intervals of equal length 9 = T/k, and define Si = iff , * - 0, . . . , k. 
Thus the sequence m* = M(s t ) will be an initial estimator for the values of the mean function at the fixed 
intervals a*. In general, however, this sequence will not satisfy the complete monotonicity assumptions of 
the model, and will thus need some modification. 

For the problem of estimating the rate function, one obtains an initial estimator from the slope of the 
mean function estimator Af(t). Specifically, the sequence 


ft = (rri* — »=!»•••.* 


will constitute a raw estimate of the failure rate function at the points 

When working with discrete, equally spaced time points, the analogue of a completely monotone function 
is a completely monotone sequence. The sequence (r t , i= 1, 2, . . .) is completely monotone if 

( — I)’' A J r, >0, j + 1 <t; j = 0,1,... ( 4 ) 


where A J is the j-th order backward difference operator 


A 0 


r. 


A 1 r, = r-j - t\_i, A J rj = A J ‘ 


A J 


- 1 . 


- 1 . 


3 > 1 


( 5 ) 


In general, the initial estimate (t u . . ., f k ) will not have the complete monotonicity property. Our goal will 
be to find the “closest” completely monotone sequence (ft, . . . , r k ), and use it as an estimate of the sequence 


2 


of failure rates at times «<. Using the criterion of weighted least squares, the problem is to find a vector r 
which minimizes 

( 6 ) 


k 

D(r,f) = 

i = 1 


i \2 


subject to the complete monotonicity constraints of (4), where w t is a set of prespecified weights. 

In practice, numerical experience indicates that the effect of the very high order difference constraints on 
the optimal solution is at most marginal; moreover, their presence leads to ill-conditioning of the optimization 
problem. Consequently we relax the constraints in (4) and consider differences of at most d (not oo), with 
d being typically 3 or 4. Similarly, it is unnecessary to constrain the sequence infinitely far into the future; 
we shall restrict the number of future intervals to l, rather than oo. Finally, it should be noted that many of 
the constraints in (4) are redundant, e.g., Ar,_x < 0 and A 3 »\ > 0 imply that Ar< > 0. Eliminating those 
redundant constraints, we finally obtain the reduced system of equations 


(-l^A 1 ^ > 0, 

(— iy'A’r fc+I > 0, 


d + 1 <t< Jfe + J 

0 <j<d-l, 


CO 


and our problem is to minimize (6) subject to (7). t 

Notice that for the case d- 1, the problem is the well known “isotone regression’ problem described 
by Barlow et al. (1972) and addressed in the reliability growth context by Campbell and Ott (1979) , and 
Nagel et al. (1984). If the last interfailure happens to come from the right tail of the interfailure time 
distribution, f k will underestimate r(T), and the monotone constraint on r will have no effect, thereby 
leading to a negative bias. Imposing the additional constraint of convexity tends to pull this estimate 
up. In most software reliability applications, a positively biased estimate of the failure rate is safer than 
a negatively biased estimate; thus, higher order constraints seem to be desirable, and the generalization of 
isotone regression to completely monotone regression an improvement. ^ , , , 

Let us return to the problem of estimating the mean function. Recalling that its first order derivative is 

a completely monotone function, and using the above, our problem is 


D(m, m) = ^ Wi(mi - m*) 2 


i= i 


subject to (— l) d *A rrii > 0, 

(-1) J A^mjb+i > 0, 

mk = n + < 5 , 


d ^ i ^ h -h f 
0 < j < d - 1 


(8) 


where mo is defined as zero. If testing stopped at a failure, (i.e., t n = T), the value 6 = 0 should be used. 
In the case of truncated testing however, the value 6 = 0.5 is a more plausible choice. Using an argument 
based on the assumption of a Poisson Process, a value 6 = 1 is also a plausible choice. 

The optimization problems presented above are linearly constrained quadratic programming problems, 
and algorithms for their solution are readily available in the literature. However, it can be shown (see Miller 
and Sofer( 1986b)) that our particular problem of least squares regression under higher order difference 
constraints becomes increasingly ill-conditioned as the problem size grows. Thus, a numerically stable 
algorithm should be employed for its solution. For a detailed description of a viable solution approach, see 

Miller and Sofer (1986b). . . , 

An additional difficulty which arises, when attempting to include monotonicity requirements into the 
future, is that the Hessian matrix, i.e., the matrix of the second order derivatives of the objective function is 
singular, since the future predictions r { and m, ( where i = A: + 1, . . . , k + l) do not appear in the objective. 
Moreover, the optimal future rate or mean estimators obtained by the least squares objective are not unique. 
In section 3, we show how to overcome the problem of singularity, by reformulating the constraints on the 
future rates (or mean function estimates) in terms of those of the past. Surprisingly, this approach also 
provides bounds - lower and upper envelopes for these future estimates. 
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3 Predictions into the Future 


As mentioned previously, formulation (8) gives rise to some computational problems, when future pre- 
dictions are requested, i.e., when l > 0. Algorithms for solving quadratic programming problems (see 
McCormick(1983)) usually require that the Hessian matrix of the objecitve function be positive-definite. 
However, the Hessian matrix of the objective for (6), ( i.e., diag (tui, . . on ^ positive semi-definite, 

and does in fact have singularities. As a result, not only will we encounter numerical difficulties when trying 
to solve the problem directly, but the optimal solution will not be unique. Indeed, any two solution vectors 
where the first k components are equal, will yield exactly the same objective value. In other words, if the 
completely monotone sequence (ri, . . . , r*) can be extrapolated / time intervals into the future, in a way that 
the resulting sequence (ri, . . rjfc+j) is completely monotone, then all such possible extrapolations will have 
the same least squares objective. In the following we show, that among all such extrapolations, there exist 
a globally highest and a globally lowest extrapolation, and all other completely monotone extrapolations 
into the future must lie in between the highest and lowest bounds. We thus have an envelope in which all 
completely monotone extrapolations are restricted. In addition, we derive the conditions under which the 
sequence (r \ , , . . , r*) can be extrapolated as a completely monotone sequence into the future. 

Consider the completely monotone sequence of order d : R = (r lf . . . , r*). The sequence (r*+i, . . r* + j) 
is defined to be a feasible completely monotone extrapolation of order d for H, if the sequence (r 1} . . . , r* + j) is 
completely monotone up to order d, i.e., it satisfies (7). In addition, this extrapolation constitutes an upper 
bound for all feasible extrapolations of order d, if any other such extrapolation, (fk+i, . . . , ffc+i) satisfies 
ffc+i < r*+i for i — 1 Similarly it constitutes a lower envelope if f k +i > f '*+» for all i. In the following, 
we derive conditions for the existence for such higher and lower envelopes for the completely monotone 
extrapolations. 

For the case d = 1 and d = 2, the sequence (rj, . . .,rt) can be extrapolated into the future by letting 
r fe+t — r*, i = 1 This extrapolation is clearly the upper envelope for all completely monotone 
extrapolations of order 1 and 2 , and is always feasible. Also for d = 1, the extrapolation = 0 is clearly a 
lower envelope for all isotone extrapolations. The next proposition shows, that the lower envelope for feasible 
extrapolations of order d = 2 is along a piecewise linear function which has slope until it reaches zero, 

after which it continues as a constant function zero. We define 


-{!■ 


-Ffc/A^Jb]) 


if A l r k > 0 
if A^fc = 0 


( 9 ) 


where [n] is the greatest integer not larger than n. 


Proposition 1 Consider the constraints (7) with d — 2 and fixed l > 0, and let (r be a feasible 
solution to (7) with / = 0. Then the extrapolation 


r k + iA x r k i = 1, — ,p 


TkH { 0 i = p+l,...,l 


( 10 ) 


is a lower envelope for all feasible extrapolations of order 2 to (ri, . . . , r*) . 


Proof: The solution above is clearly monotone, and A 2 rj(.+j = 0 for i = 1, . . . ,p and i = p + 3, . . . , /. In 
addition, A 2 r* +p + i = ~{r k + (p + l^r*) and A 2 r fc+p + 2 = r k +pA 1 r fc , which, by definition of p are both 
nonnegative. Thus the constraints of (7) for d = 2 are satisfied. Note also, that for any other feasible 
extrapolation (fjfe + i, . . . , ^fc+i) we have 

A > aV*. 


Thus, if i < p then 


fjfe+i = r k + > r* + iA'rj. = r k+l . 

j=i 


It follows that (10) is a lower envelope as proposed. 


4 


Proposition 2 Consider the constraints (7) with d — 3 and fixed l > 0. A solution (ri, . . ■ ,r k ) which 
satisfies (7) with I = 0 can he extrapolated to a vector (r lt . ,.,r k+ i) which satisfies (7) with l > 0 if and only 

r k + jA l r k + ~j(j + l)A J r fc >0; j = l,...,l. 


(11) 


In addition, let 


-U 


-A 1 r i /A 3 rjfc] if A 3 r* > 0 
if A a r* = 0 


Then the upper envelope of all feasible extrapolations for d = 3 is given by 


j r k + iA 1 r* + 51(1 + l)A 3 r t *=l,...,g 

rt+ ‘ “ \ r k+t i = q+ 1,.. 

Proof: First we note, that any feasible extrapolation satisfies 

t 

nt+i = r k + y; A 1 r* + j 
j = 1 

t j 

= + ^(AV* + 53 


;=i 


h= 1 

j 


= r fc 4- iA 1 rjk + 53 A3r *+* 

i=i h=i 

< r fc 4- lA 1 ^ -|- ii(i 4- l)A 2 r* 

and the nonnegativity of rjt + j implies that (11) must hold. Conversely, assume that (11) holds. Now since 
{r*} is completely monotone of order 3, the sequence {— A 1 ^} is completely monotone with order d = 2. 
Using Proposition 1 for the lowest feasible convex extrapolation for {— A 1 ^}, we obtain the upper envelope 
for completely monotone extrapolations of (r*i, . , . , r*) of order 3. 

Proposition 3 Consider the constraints (7) with d = 3 and fixed l > 0, and let (r*, . . rj.) be a solution to 
(7) with l = 0 satisfying ( 11 ). Let p be defined as in (9). 

(a) . If p > /, then the extrapolation 

r k +i = r k + i& l r kt i = 

is a lower envelope for all feasible extrapolations of order 3 to (r i, . . . , r*). 

(b) . If p < l, let 

u = min(/, 1 + [-2r fc /A 1 r /k ]). 


Then the extrapolation 
r fc+- 


| r k + tA 1 ^ 4- §i(t + 1) ( r>} ) *=l,...,u 

\ 0 i = u+l,...,l 


( 12 ) 


ij a lower envelope for all feasible extrapolations of order 3 to (ri, . . . , r^) . 


The proposition states that the lowest envelope is a linear function with slope A 1 ^, provided that such 
a linear function is feasible (nonnegative); otherwise it starts as a quadratic function with constant second 
order difference 

/ 2 (r k 4 uA ! r fc ) \ 

V u(u+l) J 

which flattens to zero at r fc+u , and from there continues as zero. 


a — — 
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Proof: If p > l then (10) is a feasible extrapolation of order 3, thus (a) follows from Proposition 1. 

If p < l then (10) will not satisfy the third order difference constraints. We now show that for this case the 
function of (12) is indeed a lower envelope for any feasible extrapolation. First assume that there exists a 
feasible extrapolation f k + i . . . f k+l for which A 2 f k + 1 < a. Then 

f»+i + («-l)A 1 < , fc+l + J(*-l)«A a f» + i < r t +uA 1 r t + iu(ti+l)A 3 f fc+1 

= rfc + uA 1 rt — (ft + uA 1 rj ; ) = 0, 

in contradiction to the conditions given by Proposition 2, for a feasible extrapolation for rj, . . . , r k , ^fc+i- We 
therefore conclude that any feasible extrapolation has a second order difference of at least a. If, on the other 
hand, A J f t + i > o, then f* + i > r k+1 . An inductive argument starting from r* +1 completes the proof. 

Proposition 4 Consider the constraints (7) with d = 4 and fixed l > 0. A solution (ri,...,r*) which 
satisfies (7) with l = 0 can be extrapolated to a vector (ri, . . . , r t+( ) which satisfies (7) with l > 0 if an on y 

if , 

AVj. + jA 2 r k + -j(j + l)A 3 r t <0, j - 
r k + lA l r k + -1(1 + l)A 2 ffc > 0, 

r k + |(j - l)A x rik + iy(j - l)A 2 r fc >0, j = 1,...,I. 

If A 1 r k + lA 2 r k < 0 then the upper envelope of all such extrapolations is given by 

r k+i = r k + iA l r k + ^i(i + l)A 2 r t , * = 1 


(13) 

(14) 

(15) 

(16) 


Otherwise, let 


v — min(l, 1 + [— 2A 1 r*/A 2 rj.]). 


Then the upper envelope of all such extrapolations is given by 


rfc+» 


{ 


r k + tA 1 r i + \i{i + l)A 2 r fc + ii(i + l)(t + 2) ( ^ ) 

ffc+t/ 


i = 1, • - ■ , v 

i = v + 1, .. .,1. 


(17) 


Proof- If the sequence {»■*+,} is a feasible extrapolation of order d - 4 then the sequence { A rj. + ,} is 
a feasible extrapolation of order d = 3. By Proposition 2, the conditions for existence of the latter are given 
by (13). In addition, the upper envelope of all extrapolations for d - 4 is the sequence {»■*+<} for which 
{-A 1 r fc+i } constitutes the lower envelope of all extrapolations of order d = 3. Applying Proposition 3 with 
respect to the sequence {-A 1 r ife+l } and integrating over this lower envelope yields the sequence of (16) an 
(17). Note that by construction, the resulting sequence is nonincreasing, convex with nonpositive third order 
difference. It remains to determine the conditions under which this sequence is nonnegative. First, we note 
that condition (14) guarantees that (16) will be nonnegative. From Proposition 2 this is also a necessary 
condition. Also conditions (15) guarantee that r k+v is nonnegative for any possible value of v between 1 and 
l Since (17) represents a decreasing function which becomes constant for i > v, this guarantees that r t+l is 
also nonnegative for any i. To show that conditions (15) are also necessary, define 


P(j) = rjt + 2 -{j - l)A x r fc + ij(j - l)A 2 r*. 


It is easy to see that P(j) decreases for j = 1, . . .,v and increases for j = v,...,l. Suppose that (15) is 
violated for some j. Let J be the smallest index to violate this condition. It follows that j < v and that 
P(v) < 0. This in turns implies that < 0, and thus no feasible extrapolation with d = 4 is feasible, 

hence a contradiction. This completes the proof. 

We shall now derive the envelopes for prediction for the mean function. Consider a sequence of order d. 
M = (m u ...,m k ) which satisfies (8). The sequence (m fc+1 , . . . , m k+l ) is defined to be a feasible extrapola- 
tion of order d for Af, if the sequence (mi, ... , m fcfl ) satisfies (8). In addition, this extrapolation constitutes 
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an upper bound for all feasible extrapolations of order d, if any other such extrapolation — ,m fc+ i) 

satisfies m fc+l < m fc+i for i = 1 <• Similarly it constitutes a lower envelope if > m k+i for i - 1 ■ • • , *• 

In the following, we derive conditions for the existence for such higher and lower envelopes for the feasible 
extrapolations for M. We note that the derivative of the mean function is a completely monotone function. 
Therefore, the lower and upper bounds for all feasible extrapolations of order dto ro 1 , ...,m l are obtaine 
by integrating respectively over the lower and upper bounds for all feasible extrapolations of order d - 1 to 

A ^Consequ en 1 1 y , for the case d = 1, d = 2 and d = 3, the sequence (rm, ...,m k ) can always be extrapolated 
into the future. The upper envelope for all feasible extrapolations of order up to 3 is the linear function 

mjk+t = m k + iA 1 m k - 

For d - 1 and d = 2 the extrapolation = m k is clearly a lower envelope for all feasible extrapolations. 

The next proposition shows, that the lower envelope for 
quadratic which tapers off to a constant function. 

Proposition 5 Consider the constraints (8) with d = 3 
solution to (8) with l = 0. Let 

[-A l m k /A 2 m k ] 

/ 



feasible extrapolations of order d — 3 is along a 


and fixed l > 0, and let (r u . . . , r h ) be a feasible 


if A 3 mjb > 0 
if A 2 mjt = 0 


(18) 


Then the extrapolation 


m k+i = 


{ 


m k + + it(t + l)A 3 mj. 

m k+p 


»= l,...,p 
i = p+ 1,...,/ 


is a lower envelope for all feasible extrapolations of order 3 to (m 1( . . . , m k ) . 


Proof: Follows from Proposition 1. 

Proposition 6 Consider the constraints (8) with d = 4 and fixed l > 0. A solution (mi, . . ,m*) which 
satisfies (8) with l = 0 can be extrapolated to a vector (m k , . . . ,m k+t ) which satisfies (8) with l > 0 if and 


only if 

A 1 m k + jA 2 m k + ^j{j + l)A 3 m fc >0, j = 1, .... I. 

(19) 

Let 

f [— A J mi/A 3 mt] if A 3 m t > 0 



9 ~ \ l if A 3 m k = 0 


and let 

a = A 1 m k + qA 2 m k + ^q(q + l)A 3 mjt 



Then the upper envelope of all such extrapolations is given by 

_ j m k + iA l m k -f |i(i + l)A 2 mfc + | »(» + l)(i + 2 )A 3 tth i = 1, • • • , 1 
mt+ * — \ m k+ ,+{i-q)a i — q + l,...,l. 


Proof: Follows from proposition 2. 

Proposition 7 Consider the constraints (8) with d = 4 and fixed l > 0, and let (mi, . . . , m*) be a solution 
to (8) with l - 0 satisfying (19). Let p be defined as in (18). 

(a). Ifp > l, then the extrapolation 

mjt-i-i = rn k + iA l m k -f \ 4- 1) A 2 m k , i = 1, . . . , p 


is a lower envelope for all feasible extrapolations of order \ to (tth, . • • , rn k ^. 

(b). If p < l, let 

u - min(/, 1 + [-2A 1 m Jb /A m k }). 
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Then ihe extrapolation 


m„ + iA'm, + i«(i + l)A’m, + |i(i + 1)(< + 2) , i = 1 » 

* = u+l,...,f 

mjb+u 


is a 


m k +i - | 

lower envelope for all feasible extrapolations of order 4 to (mi, ...,m k ) ■ 


The proposition states that the lowest envelope is either along a quadratic function, or it starts as a cubic 
function which tapers off to a constant function. Its proof follows from Proposition 3. 


4 Monte Carlo Study of Performance 

To get an idea of how well future prediction envelopes estimate future behavior, we conducted a small Monte 
Carlo simulation experiment. Our goal is to estimate the number of events over some finite horizon. As 
in Miller and Sofer (1986a), we compare the completely monotone approach to some of the more popular 
parametric models. A value of d = 4 is used for the completely monotone estimation (6 is taken as 1). Thus 
the least squares problem (8) is solved for d = 4, with the constraints of (19) replacing the constraints of (8) 
for * = k + 1 ...,k + l. Propositions 6 and 7 are applied to the resulting solution to obtain the upper and 
lower envelopes for the future mean function. Finally, we need a point estimate of the expected number of 
failures. We have arbitrarily decided to use the midpoint of the envelope. . 

Our choice of parameter models consists of three families of nonhomogeneous Poisson processes (NH )• 

The mean functions of the NHPP’s may have exponential, power or logarithmic form: 


M txp (t) = 

Mpow (0 


7(1 -e-*), 
yt a } 

7 log {/3t + 1) 


Those models are fit to data by using the method of maximum likelihood as described by Musa and Oku- 
moto(1984). Furthermore, we define a fourth model which is a mixture of the above three. It is fit by 
selecting the best fitting (i.e., maximum likelihood) of the three models. This is the best parametric 
model, among the three possibilities. . , rn inn! 

We draw our data from 16 different Poisson processes. Each process is observed over the interval [0,100] 
and the future interval is [100,125], i.e., 25% into the future. We used k = 20 and l - 5. The 16 cases 
provide a variety of different growth patterns. Each case is replicated 400 times. The cases are summarized 

11 ^The* performance of the parametric models and the completely monotone approach are summarized in 
Tables 2 3 and 4. Table 2 shows the average prediction made by each model for the 400 replicates of each 
case Table 3 shows the average percentage error, or bias. Table 4 shows the root- mean-square percentage 
error for the 400 estimates made by each model for the 16 test cases. We note that when the data comes 
from a certain model, then that particular model gives the best predictions. However in most cases, the 
completely monotone comes in as “second best”, i.e., it gives better predictions than those given by using 
the incorrect parametric model. In practice, of course, it is highly unlikely that a parametric model used 
for prediction will indeed be the “correct” model from which the failure data was generated In Table 5 the 
performance of the prediction envelopes is summarized. Note that the majority of the envelopes have zero 
width i.e., the upper envelope is identical to the lower envelope. 

In conclusion, we stress that some components in the formulation of the completely monotone model were 
chosen arbitrarily. Other definitions of the raw estimates and other objective functions will give differen , 
and possibly better estimates. Nevertheless, the completely monotone approach shows a robustness not 
exhibited by the individual parametric models. The procedure has quite low bias which is less than that 
caused by using the incorrect parametric models for prediction. Comparisons to the best parametric mode 
are unfair because the Monte Carlo data is, in effect, drawn from that model. We could use other models 
to generate data for which this “best” parametric model is inferior to the more robust completely monotone 

approach. 


Model 

Number 

Type of 
NHPP 

Parameter 

M(125) - 
M(100) 

1 

Homogeneous 


10.00 

2 

Power 

a =.749 

7.28 

3 

Power 

a =.557 

5.29 

4 

Power 

a =.410 

3.83 

5 

Power 

a =.296 

2.73 

6 

Power 

a =.208 

1.90 

7 

Logarithmic 

/ 3 =.0124 

6.42 

8 

Logarithmic 

0 =.0429 

4.43 

9 

Logarithmic 

0 =.131 

3.16 

10 

Logarithmic 

0 =.461 

2.27 

11 

Logarithmic 

0 =2.43 

1.62 

12 

Exponential 

rj =.00808 

5.88 

13 

Exponential 

t) =.0167 

3.17 

14 

Exponential 

t) =.0265 

1.47 

15 

Exponential 

7) =.0385 

0.54 

16 

Exponential 

7] =.0550 

0.12 


Table 1: Data Models (Poisson Processes). All models are scaled so that E(N{ 100)) - Af(100) 


Model 

Number 

True 

Mean 

EXP 

LOG 

POW 

BEST 

CM. 

Mdpt. 

1 

10.00 

8.67 

8.86 

9.33 

8.73 

9.52 

2 

7,28 

5.62 

6.11 

7.34 

6.38 

7.72 

3 

5.29 

2.97 

3.73 

5.36 

4.70 

5.88 

4 

3.83 

1.36 

2.23 

3.88 

3.62 

4.50 

5 

2.73 

0.51 

1.31 

2.76 

2.65 

3.40 

6 

1.90 

0.15 

0.75 

1.92 

1.87 

2.52 

7 

6.42 

6.10 

6.76 

8.20 

6.48 

7.44 

8 

4.43 

3.41 

4.68 

6.63 

4.19 

5.38 

9 

3.16 

1.64 

3.29 

5.27 

2.89 

4.06 

10 

2.27 

0.64 

2.35 

4.11 

2.28 

3.08 

11 

1.62 

0.18 

1.66 

3.10 

1.71 

2.30 

12 

5.88 

5.86 

6.57 

8.09 

6.26 

7.31 

13 

3.17 

3.21 

4.70 

6.72 

3.67 

4.73 

14 

1.47 

1.51 

3.62 

5.63 

1.91 

2.84 

15 

0.54 

0.57 

2.95 

4.77 

0.75 

1.60 

16 

0.12 

0.14 

2.48 

4.07 

0.21 

0.88 


Table 2: Average Predictions of Mean Number over Future Horizon 




Model 

Number 

Fitted Model 

EXP 

LOG 

POW BEST 

CM 

1 

-13. 

-11. 

-7. 

-13. 

-5. 

2 

-23. 

-16. 

+i. 

-12. 

+6. 

3 

-44. 

-30. 

+i. 

-11. 

+ 11. 

4 

-65. 

-42. 

+i. 

-6. 

+ 17. 

5 

-81. 

-52. 

+i. 

-3. 

+24. 

6 

-92. 

-61. 

+i. 

-2. 

+33. 

7 

-5. 

+5. 

+28. 

+i. 

+ 16. 

8 

-23. 

+5. 

+50. 

-6. 

+21. 

9 

-48. 

+4. 

+67. 

-8. 

+29. 

10 

-72. 

+3. 

+81. 

0 . 

+36. 

11 

-89. 

+3. 

+98. 

+6. 

+42. 

12 

0 . 

+ 12. 

+32. 

+6. 

+24. 

13 

+ 1. 

+48. 

+ 112. 

+ 16. 

+49. 

14 

+3. 

+ 146. 

+282. 

+30. 

+93. 

15 

+6. 

+448. 

+788. 

+40. 

+ 199. 

16 

+ 14. 

+ 1921. 

+3216. 

+74. 

+615. 


Table 3: % Prediction Error (Bias) for Mean Future Number 


Model 

Number 

Fitted Model 

EXP 

LOG 

POW BEST 

CM 

1 

26. 

24. 

29. 

24. 

29. 

2 

39. 

32. 

23. 

31. 

30. 

3 

53. 

39. 

23. 

32. 

39. 

4 

69. 

46. 

23. 

29. 

48. 

5 

83. 

55. 

23. 

26. 

60. 

6 

93. 

62. 

23. 

26. 

73. 

7 

37. 

32. 

39. 

36. 

38. 

8 

44. 

31. 

59. 

43. 

50. 

9 

57. 

26. 

75. 

47. 

62. 

10 

75. 

23. 

89. 

40. 

74. 

11 

90. 

21. 

99. 

27. 

85. 

12 

38. 

34. 

47. 

37. 

45. 

13 

45. 

61. 

120. 

53. 

80. 

14 

54. 

155. 

292. 

82. 

142. 

15 

67. 

461. 

805. 

134. 

278. 

16 

95. 

1956. 

3268. 

336. 

776. 


Table 4: % Root Mean Square Error for Mean Future Number Predict 




f Data 
Model 
Number 

Fraction 

Zero 

Width 

Fraction 

Non-zero 

Width 

Av. width 
Non-zero 
Envelope 

True Mean Coverage 
Fraction Fraction Fraction 

Overestimate Correct Underestimate 

1 

.715 

.285 

0.348 

.408 

.067 

.bib 

2 

.515 

.485- 

0.816 

.575 

.105 

.320 

3 

.503 

.497 

0.966 

.542 

.182 

.275 

4 

.548 

.452 

0.759 

.550 

.167 

.283 

5 

.570 

.430 

0.588 

.585 

.160 

.255 

6 

.600 

.400 

0.414 

.595 

.155 

.250 

7 

.420 

.580 

1.116 

.610 

.193 

.197 

8 

.417 

.583 

1.334 

.515 

.283 

.202 

9 

.505 

.495 

0.897 

.560 

.243 

.197 

10 

.573 

.427 

0.616 

.600 

.193 

.208 

11 

.573 

.427 

0.441 

.605 

.182 

.213 

12 

.363 

.637 

1.228 

.648 

.190 

.162 

13 

.305 

.695 

1.828 

.520 

.400 

.080 

14 

.321 

.679 

1.300 

.574 

.333 

.093 

15 

.503 

.497 

0.656 

.652 

.243 

.105 

16 

.698 

.302 

0.209 

.925 

.063 

.013 


Table 5: Performance of Completely Monotone Prediction Windows 
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